Bayesian Effective Samples and Parameter Size

Suppose we have a posterior density for a parameter given a sample and we form a second posterior density for the same parameter, based on a different model and a different data set. Then we can evaluate the relative entropy distance between the two posteriors. Minimizing the relative entropy over the second sample gives the virtual sample that would make the second posterior as close as possible to the first in an inferential sense. For instance, if the first posterior is based on a dependent dataset and the second posterior is based on an independence likelihood, the optimization transfers the effective inferential power of the dependent sample into the independent sample. We present further examples of this type of optimization for models with nuisance parameters, finite mixture models and models for correlated data. Finally, we use our approach to choose the effective parameter size in a Bayesian hierarchical model.